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Abstract 

Background: Integrative medicine (IM) is a patient-centered, liealing-oriented clinical paradigm tliat explicitly 
includes all appropriate therapeutic approaches whether they originate in conventional or complementary medicine 
(CM). While there is some evidence for the clinical and cost-effectiveness of IM practice models, the existing evidence 
base for IM depends largely on studies of individual CM therapies. This may In part be due to the methodological 
challenges inherent in evaluating a complex intervention (i.e., many interacting components applied flexibly and 
with tailoring) such as IM. 

Methods/Design: This study will use a combination of observational quantitative and qualitative methods to 
rigorously measure the health and healthcare utilization outcomes of the University of Arizona Integrative Health 
Center (UAIHC), an IM adult primary care clinic in Phoenix, Arizona. There are four groups of study participants. The 
primary group consists of clinic patients for whom clinical and cost outcomes will be tracked indicating the impact 
of the UAIHC clinic (n = 500). In addition to comparing outcomes pre/post clinic enrollment, where possible, these 
outcomes will be compared to those of two matched control groups, and for some self-report measures, to regional 
and national data. The second and third study groups consist of clinic patients (n = 1 80) and clinic personnel 
(n = 15-20) from whom fidelity data (i.e., data indicating the extent to which the IM practice model was implemented 
as planned) will be collected. These data will be analyzed to determine the exact nature of the intervention as 
implemented and to provide covariates to the outcomes analyses as the clinic evolves. The fourth group is made up of 
patients (n = 8) whose path through the clinic will be studied in detail using qualitative (periodic semi-structured interviews) 
methods. These data will be used to develop hypotheses regarding how the clinic works. 

Discussion: The US health care system needs new models of care that are more patient-centered and empower 
patients to make positive lifestyle changes. These models have the potential to reduce the burden of chronic 
disease, lower the cost of healthcare, and offer a sustainable financial paradigm for our nation. This protocol has 
been designed to test whether the UAIHC can achieve this potential. 

Trial registration: Clinical Trials.gov NCTOl 785485. 

Keywords: Integrative medicine, Observational study. Study protocol. Fidelity monitoring. Implementation 
fidelity, Complex interventions. Comparative effectiveness research. Healthcare utilization. Cost outcomes. 
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Background 

This article describes the protocol for a two-year study 
of the health and economic impact of a primary care in- 
tegrative medicine (IM) clinic. IM is defined as a health 
promoting, prevention-oriented clinical paradigm that 
explicitly includes conventional medical treatment to- 
gether with the use of complementary medicine (CM) 
modalities that have sufficient evidence and safety for 
use [1-5]. Practice models in IM are emerging [6], but 
most of the models provide consultative or specialty 
care; few provide primary care. There are also few meth- 
odologically valid health services research studies on the 
clinical and cost effectiveness of IM practice models, 
particularly for primary care [7-12]. 

The evaluation of a IM primary care model presents a 
number of methodological challenges. Some are the re- 
sult of the evolving nature of the concept of IM [12], 
and most have been discussed in the literature to illus- 
trate the complex needs of CM research [13-20]. In gen- 
eral, these challenges apply to research involving any 
complex intervention (e.g., a new care delivery model) 
[21-24]. Each challenge is discussed below accompanied 
by recommended solutions. 

Complex, individualized intervention 

Although IM is defined as both a clinical treatment 
paradigm and a practice organization model, the existing 
evidence base for IM depends largely on studies of the in- 
dividual components [7,8,10,14,25]. While IM combines 
conventional and evidence-based CM modalities (having 
practitioners who can offer these CM modalities is an im- 
portant aspect of the practice organization model), it is 
also a larger clinical paradigm for a process of whole per- 
son, patient-centered, healthcare that embraces the innate 
healing capacity of the human organism and attends to 
the importance of lifestyle [2,26,27]. Whether seen as a 
clinical paradigm or as a practice model, IM meets the 
definition of a complex intervention. It consists of a num- 
ber of interacting components that make up and affect 
care, and these components are applied with a large de- 
gree of flexibility and tailoring [28] . To create an evidence 
base for IM itself, studies that characterize and evaluate 
the outcome of intact IM clinic practices are needed. 

Several researchers have recommended that complex in- 
terventions simply be studied as "black boxes" [14,16,19,20]; 
as interventions whose outputs can be measured, but 
whose internal mechanisms are unknown. Since subjects 
can be randomized to one black box or another, no new 
methods are required. Others recommend that the boxes 
be "unpacked" — that every element in the intervention be 
defined and its outcomes measured [29,30]. The "black 
box" approach offers no assurance that the box contents 
are a valid representation of the intervention; however, 
given the number of different treatments and potential 



diagnoses, complete "unpacking" may prove an insur- 
mountable measurement, analysis, and interpretation 
challenge. The concept of implementation fidelity offers 
a balance between these two extremes. Outcomes are 
still measured for, and attributed to, the intervention as 
a whole (the "box"). However, the contents of the box 
are defined and measured to document what was actu- 
ally delivered — i.e., what the box contains. 

When a study compares two drugs (or a drug to pla- 
cebo), all else is held (or assumed to be) equal - in part 
through randomization, and in part through strict study 
protocols, including participant and practitioner blind- 
ing. As both comparison groups are being treated within 
the same practice model (e.g., conventional primary 
care), little, if any, effort is given to defining that model. 
Faithfulness to the intended intervention, defined as 
Intervention implementation fidelity, is ensured through 
monitoring adherence to protocols. The objective is to 
ensure that the only difference between groups is receipt 
of the drug under study [22]. 

In contrast, when two practice models are compared 
in a study, very little can be held equal. The study proto- 
col is by necessity less prescriptive as to what is deliv- 
ered, when, how, and by whom. The constraints placed 
upon possible interventions are loosened to allow practi- 
tioner autonomy in individualizing patient care. Not only 
may different treatments be offered, but different practi- 
tioners with different training may diagnose and then 
prescribe those treatments with different clinical goals 
and under different philosophies of care. Some method 
of defining the practice models is needed to capture and 
measure differences to ensure that interventions are im- 
plemented as intended, and to know what is actually be- 
ing compared. 

The concept of intervention implementation fidelity 
emerged in the 1960s in the field of psychotherapy when 
early attempts to sort out design and interpretive problems 
in outcomes studies proved impossible. For an overview 
of fidelity and its measurement, see Bond et al. [31]. Since 
the 1960s, including measures of intervention fidelity has 
become a standard requirement for research in a number 
of applied fields, including medicine [21,22,24,28,32,33]. 
Fidelity monitoring is important for the interpret- 
ation of study results (e.g., avoidance of Type III 
error - erroneously concluding that an intervention had 
no impact when it was actually not implemented), and for 
replication [22,23,32]. Fidelity measures can also be used 
as covariates in the analyses to determine treatment effect 
and avoid invalid statistical conclusions [24] . 

Assessment of implementation fidelity will help with 
defining 'what was measured' in complex interventions. 
However, to understand the 'why it happened' causal ef- 
fects of IM, a number of researchers have recommended 
the use of mixed methods — e.g., including qualitative 
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methods in the study design [13,15,20,21,34]. Randomized 
controlled trials (RCTs), even when coupled with advanced 
quantitative methods, cannot explain complex causal re- 
lationships [20,21]. Well-designed multi-method studies 
allow a deeper understanding of the mechanisms that 
might be at work in the intervention [21]. 

Whole person outcomes as markers of effectiveness 

IM's focus on treating the whole person, is believed to 
have outcomes beyond those associated with treatment 
of the targeted condition [25,35]. The emphasis on self- 
care, the benefits of a good patient-practitioner relation- 
ship, and the promotion of the body's self-healing capacity 
are all believed to result in enhanced health [35]. In par- 
allel to the concept of treating the whole person is the 
concept of using a whole system of healthcare services 
delivery. In IM this includes a focus on the patient- 
provider relationship and a full range of potential treat- 
ments. Together these result in the need to measure 
outcomes across a number of dimensions (e.g., spiritual, 
social, physical, mental, emotional) [29,36]. The Federal 
Coordinating Council for Comparative Effectiveness 
Research (CER) has stressed the importance of "assessing 
a comprehensive array of health-related outcomes..." [37] 
to provide the information that patients, clinicians and 
policy makers need to make good healthcare decisions. 

Patient preferences and the infeasrbllity of randomization 

Patients are rarely neutral about IM. They typically in- 
dicate strong preferences for or against this approach to 
healthcare, which makes it hard to accrue patients in 
randomized trials [14]. In the absence of the ability to 
randomize appropriately, observational approaches are 
recommended [16,18]. Observational studies are also 
recommended for complex systems [19] and to capture 
actual practice [29]. When well-designed, they have 
been found to produce results comparable to RCTs, 
neither over- nor underestimating treatment effects 
[38-40]. CER methods allow for studies of all types of 
health-related interventions, including delivery systems 
[21], and recognize observational studies as valid de- 
signs [41-43]. 

The protocol described in this article addresses the 
clinical and economic evaluation of an IM primary care 
clinic. Although the outcomes and processes of care of 
this particular clinic, as defined in its business plan, are 
being evaluated, it is anticipated the findings from this 
study will contribute to IM health services research in 
general. Because IM practice is a complex intervention, 
the study design includes fidelity monitoring and evalu- 
ation, and an observational design that includes qualitative 
data collection and analysis. And because IM practice 
offers whole-person care, a broad range of outcomes are 
being measured. 



Methods/Design 

This two-year study is Phase II of a two-phase, three- 
year project. The purpose of Phase I was to design the 
protocol for the Phase II study, including clarifying the 
design of the IM clinic model to be tested and identifying 
the measures to use for outcomes and to monitor treat- 
ment fidelity. The purpose of Phase II is to conduct a 
prospective outcomes evaluation of the clinical and cost 
effectiveness of an IM professional practice model for 
adult primary care. 

Study design 

A combination of observational quantitative and qualita- 
tive study designs will be used to measure the health 
and healthcare utilization outcomes of a primary care 
IM clinical model. Where possible, the clinical and eco- 
nomic impacts seen in this IM model will be compared 
to data available from one or more comparison groups 
(see discussion of comparison groups below). When 
comparison group data are not available for outcomes — 
e.g., for most of the self-report data and for the medical 
records data — baseline values for clinic participants will 
be compared to those measured during and after treat- 
ment. Table 1 shows the types of comparisons planned 
for each outcome data category for the observational 
study. 

Because the IM professional practice model is a com- 
plex intervention, implementation fidelity will be assessed, 
in addition to clinical and economic outcomes. Fidelity 
data (gathered on patients' and practitioners' experiences 
and from medical record data) will serve four purposes. 
First, these data will be used as a measure of intervention 
integrity — i.e., was the intervention/practice delivered as 
intended. Second, because integrative primary care has 
not previously been defined and evaluated, and since the 
clinic under study is new, and thus continuously develop- 
ing and modifying operational and treatment protocols 
and procedures, these fidelity data will also describe what 
happened. Third, data on operations and how they may 
have evolved over time are important as potential covari- 
ates in the analysis of outcomes, for proper interpretation 
of results, and for external validity and replication. Finally, 
fidelity data provide quality control feedback to clinic staff 
as to how the IM clinic model is being implemented by 
staff and perceived by patients. This will allow for quality 
improvements of the IM primary care clinic over time. 

This study received the approval of the University of 
Arizona Institutional Review Board, and is registered in 
clinicaltrials.gov (NCT01785485). 

Setting and Participants 

Data used in the evaluation will be gathered from eli- 
gible and consenting participants who are also patients 
enrolled at the University of Arizona Integrative Health 
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Table 1 Outcomes measured for the UAIHC 

Frequency of Pre/post Comparisons 

measurement comparisons against 

control 
group(s) 

Healthcare One year prior and one X X 

utilization and year post-study enrollment 

costs 

Diet quality BL, 3, 6, and 12 months X X" 

Exercise BL, 3, 5, and 12 months X X*" 

Patient 3 6, and 12 months X X™ 

satisfaction 

Self-report BL, 3, 6, and 12 months X 

health 

outcomes 

• Self-efficacy 

• Stress 

• Sleep quality 

• Pain 

• Fatigue 

• Depression 

• Anxiety 

• Health-related 
quality of life 

• Well being 

• Work 
productivity 

• Condition- 
specific 

• outcomes 

Medical records 3, 6, and 12 months X 
data 

BL = baseline (clinic enrollment). 

*Primary outcome; comparison is to two matched control groups. 
**Comparison here is to national (and where available statewide) averages 
from the Center for Disease Control and Prevention's Behavioral R\sk Factor 
Surveillance System (BRFSS) survey [511. 

"""Comparison here is to national and regional norms using the Consumer 
Assessment of Healthcare Providers and Systems (CAHPS) Adult Visit Survey 
2.0 data [50). 

Center (UAIHC), an integrative medicine adult primary 
care clinic in Phoenix, Arizona. Operated in conjunction 
with District Medical Group, Inc., UAIHC is financed 
through insurance reimbursements and a patient mem- 
bership fee structure. At present, two major employers 
in the Phoenix area pay a portion of the membership fee 
for their employees who elect UAIHC as their primary 
care provider. There are four groups of study partici- 
pants. The first (and main) group consists of eligible par- 
ticipants from the clinic patient population for whom 
clinical and cost outcomes will be tracked indicating the 
impact of the UAIHC clinic (n = 500). The second and 
third groups consist of clinic patients (n = 180) and clinic 
personnel (n = 15-20) from whom fidelity data will be 
collected. The fourth group is made up of the patients 



(n = 8) whose path through the clinic will be studied in 
detail using qualitative methods. The clinic patients in- 
volved in the outcomes study sample may or may not be 
included in the fidelity patient sample, and vice versa. 
However, the qualitative data sample is expected to be a 
subset of the outcomes sample. 

Outcomes sample 

Only patients who are members at UAIHC and who 
meet the inclusion/exclusion criteria are eligible to en- 
roll in this study. Participation in the study will continue 
for at least one year after enrollment. It is anticipated 
that at least 500 patients will consent to participation 
during the first nine months of the study. Our primary 
outcome for our main analysis is change in total costs 
(see the Analysis section below). Using the standard de- 
viation seen in a set of healthcare utilization data from a 
related group, a sample size of 200 patients should be 
sufficient to detect a small effect size (Cohen's d = .20) 
difference in total costs at a .80 power using an alpha 
of .05. Therefore, our expected sample size of 500 may 
also allow for some subgroup analysis. Inclusion criteria: 

1) Enrollment in primary care (members) at UAIHC; and, 

2) adults ages 18 or older. Exclusion criteria: 1) currently 
pregnant at time of recruitment; 2) patients who attend 
the UAIHC clinic as "consultation-only" patients; and, 

3) significant cognitive impairment to the extent that 
the individual is unable to understand the consent and 
respond to questionnaires. 

All new patients will be advised by their primary care 
physician that they may be eligible to participate in a 
study to evaluate the effectiveness of the UAHIC. Prior 
to checkout, patients have an exit session with the 
University of Arizona (UA) Study Coordinator. During 
this session (or if necessary, through a follow-up phone 
call), the Study Coordinator will explain the nature of 
the study and make a determination of study eligibility, 
and if appropriate, consent the patient. Patients who joined 
the UAIHC before study start (July 2013) will also have 
the opportunity to participate. The study will be explained 
at their first UAIHC visit after study start. 

Fidelity patient sample 

Participants in the fidelity study will include both UAIHC 
patients and members of the UAIHC clinic team. On fi- 
delity assessment sample days (a single randomly chosen 
day of the last week of the data collection month; see data 
collection procedures below), participants will be recruited 
and enrolled from all patients who enter the UAIHC wait- 
ing room. It is anticipated that approximately 180 patients 
will consent to participation across the two-year study 
period. The inclusion/exclusion criteria for this sample are 
identical to those used for the outcomes sample with the 
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exception that patients who are currently pregnant will be 
allowed to participate in this sample. 

At check in on the fidelity data collection day, front 
desk staff will make a printed handout available describ- 
ing a "How is UAIHC Doing?" evaluation, and asking all 
patients seen that day to provide feedback through an 
online questionnaire after that day's visit. Interested pa- 
tients will supply their e-mail address and check a box 
on the printed handout expressing permission to receive 
the online questionnaire. The UA Study Coordinator will 
send all interested patients an e-mail with instructions 
and a link to the secure URL address for the fidelity in- 
formed consent form and the Web-based questionnaire. 
Patients preferring to complete the questionnaire on paper 
will have the opportunity to receive a hard copy from the 
front desk (or by mail) with instructions to place com- 
pleted questionnaires in a secure box at the clinic (or to 
return them using the enclosed self-addressed stamped 
envelope). Digitally-signing the informed consent form, 
or returning a signed paper-based consent form will 
constitute completion of the informed consent process 
for fidelity. Patients who complete the questiormaire within 
10 days will receive a $10 gift card. 

Fidelity clinic personnel sample 

The UA Study Coordinator will enroll UAIHC personnel 
at study start by sending an e-mail with a link to the se- 
cure URL address and password information for the in- 
formed consent form. Assessments will be on the same 
schedule as for patient fidelity (see Data collection below). 
A total of 15-20 clinic personnel (the entire UAIHC staff) 
will be enrolled in the fidelity study. The same consent 
process and incentive will be used as for the fidelity pa- 
tient sample. 

Qualitative/case study sample 

The qualitative study sample will be a subset of the out- 
comes sample. In addition to the inclusion and exclusion 
criteria of the outcomes sample subjects will consist of 
members who joined the study during its first three 
months. The qualitative study will be introduced to pa- 
tients by the UA Study Coordinator during the same 
meeting in which the outcomes study is discussed. Par- 
ticipation will require an additional consent, and will en- 
tail a 20-30 minute semi-structured, taped interview 
immediately following each regular outcome data collec- 
tion points. 

Intervention 

The IM professional practice model for primary care 
utilizes an inter-professional, physician-led, integrative 
clinical team to provide patient-centered care. UAIHC 
services include conventional medical care, lifestyle in- 
terventions, and evidence-based CM interventions. In 



particular, services include longer visits with providers, 
a commitment to health, a partnership agreement with 
the health care team, increased access via telephone and 
email in addition to in-person visits and classes, and a 
wider range of services including nutritional counseling, 
acupuncture, chiropractic care, stress-reducing mind-body 
approaches, and health coaching. In addition to individual 
appointments, group visits and health promotion classes 
are offered (e.g., nutrition, yoga, meditation). Emphasis is 
placed on motivating and supporting lifestyle behavior 
change. Although patients with a wide variety of diagnoses 
will be eligible and included in the study, outcomes for pa- 
tients with the following five conditions (low back pain, 
fibromyalgia, other musculoskeletal conditions, metabolic 
syndrome and diabetes, and cardiovascular disease) will 
be analyzed separately where possible. 

The UAIHC business model uses a hybrid financing 
approach that utilizes usual insurance reimbursements 
supplemented by a patient membership fee. The mem- 
bership fee covers a certain number of CM practitioner 
visits, health coaching visits, and classes (a "class" may 
comprise a series of individual sessions). Three fee tiers 
have been established: 1) basic bundle (5 visits with the 
patient's choice of CM practitioners, 2 visits with a 
health coach, and 2 groups or classes); 2) core (10 visits 
with the CM practitioners of choice, 4 health coach 
visits, and 4 groups or classes); and, 3) expanded (20 
visits with CM practitioners, 8 visits with a health coach, 
and 8 groups or classes). For some members, this fee is 
partially offset by collaborating employers who offer 
UAIHC as an employee health benefit option. (Grant 
funding supports the costs of this study for those partici- 
pants employed with these collaborators). 

According to its business plan, the IM primary care 
practice model will follow IM principles [44] as well as the 
principles of primary care [45,46] and the patient-centered 
medical home [47]. More detail on the IM primary care 
practice model is presented in Dodds et al [48] . 

Comparison group(s) 

Data from UAIHC patients will be compared, where 
possible, to those of patients seen in other clinics. The 
primary outcome, total healthcare utilization, will be 
obtained from healthcare claims data for patients whose 
employers have given access to those data. For these pa- 
tients, matched comparison groups will be developed 
using two different matching algorithms applied to de- 
identified claims data for non-UAIHC patients. One will 
use a straight matching algorithm based on age and gen- 
der, and the other will use propensity scores to match 
[49]. At least one of the participating employers also has 
employee lab values available. Where these data are 
available, the same matched comparison groups will be 
used to compare changes in key lab values seen across 
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groups. Finally, several of the self-report measures have 
national and regional comparisons available. The mea- 
sures of patient experiences and patient satisfaction are 
drawn from the Consumer Assessment of Healthcare 
Providers and Systems (CAHPS) Adult Visit Survey 2.0 
[50]. The CAHPS website contains data on the results 
of these questions nationally, by region, and by phys- 
ician specialty. The results seen for UAIHC participants 
will be compared to those seen by all providers nation- 
ally, by all providers in the West region, and by family 
practice doctors nationally. The measures of diet and 
exercise quality come from the Center for Disease Control 
and Prevention's Behavioral Risk Factor Surveillance 
System survey [51]. National and statewide averages are 
available for these items. Unfortunately, no other second- 
ary data sources are available for comparisons. For all 
other outcomes, UAIHC participants will act as their own 
controls by comparing their outcome measures over time 
to baseline. 

Data Collection 

For the outcomes evaluation, data will be collected from 
patient charts and self-reports at four intervals — baseline 
(initial clinic visit), and at 3-, 6-, and 12-month follow 
up periods. Participants will receive a reminder email 
one week before each data collection due date. Baseline 
data will be due within one week of enrollment. In the 
email they will be directed to a secure website where 
they can access and complete the outcomes data collec- 
tion instrument. If participants have not entered their 
data within 3 days of their due date, they will receive an- 
other reminder email, and those who have not entered 
their data within another 3 days of their due date will re- 
ceive a call from the UA Study Coordinator. Healthcare 
utilization and cost data from healthcare claims will be 
obtained from their employer with study participant 
consent. These data will be collected for one year before 
and one year after clinic enrollment approximately 
3 months after the 12-month follow up period to allow 
time for all incurred claims to be recorded. De-identified 
claims data for the comparison groups will be collected 
for the same period of time with "baseline" being defined 
by the average start date of study participants. 

Because the intervention may change over time as 
clinic operations start up and mature [21], fidelity moni- 
toring data will be collected monthly from study start 
for 6 months, then quarterly for 6 months, and then 
semi-annually. Patient-reported fidelity data will be gath- 
ered on a single randomly chosen day of the last week of 
the data collection month from all patients seen at the 
UAIHC clinic that day. This day will be selected by the 
UA research team using a simple computer generated 
randomization scheme and delivered to the UA Study 
Coordinator the week before the data collection week. A 



self-administered patient version of the "How is UAIHC 
doing?" questionnaire will be used for patients. For clinic 
personnel, fidelity data will be collected during the last 
week of each fidelity data collection month using an on- 
line self-administered practitioner and staff version of 
the "How is UAIHC doing?" questionnaire. Data from 
audits of randomly selected, de-identified patient charts 
and of administrative records will also be used to assess 
indicators of model implementation. 

Qualitative data in the form of the results of 20-30 minute 
semi-structured interviews will be collected by phone or 
in-person approximately every three months, and at other 
significant points in time if the patient agrees — e.g., if the 
patient chooses to leave the clinic or there is a dramatic 
change in the patient's use of clinic services. The focus is 
to collect real-time data on what the patient believes is 
the cause of any changes (or lack of change) seen in 
their health. 

Outcome measures 

Data on clinical and economic outcomes will be col- 
lected from three main sources: self-report instruments, 
chart abstraction, and health plan administrative data 
(healthcare utilization and, where available, lab values) 
from participants' employers. Because integrative medi- 
cine approaches treat the whole person, rather than sim- 
ply targeting a specific disease or symptom, they can 
have broad impacts on health [25,29,35,36]. Therefore, a 
broad set of clinical and economic outcomes for the 
UAIHC will be captured (Table 1). These are each dis- 
cussed below, starting with the self-report measures. All 
are collected at baseline (some directly and some as part 
of patients' initial clinic intake form), and then 3, 6 and 
12 months and semi-annually thereafter, unless other- 
wise noted. 

Self-report Instruments - all patients 

In addition to general socio-demographic data and an 
item on patients' expectations of the clinic (both asked 
only at baseline), several factors are captured that could 
be seen as overall determinants of health and/or inter- 
mediate outcomes: diet, exercise, self-efficacy, stress, 
sleep and patient satisfaction. In addition, a number of 
specific (pain, fatigue, depression and anxiety) and global 
(quality of life, well-being, and work productivity) out- 
comes are captured. Each is described briefly below. It is 
estimated that it will take patients 15-20 minutes to 
complete these instruments. 

• Diet quality. Diet quality is assessed by the Fruits 
and Vegetables module of the Behavioral Risk Factor 
Surveillance System [51]. Six items assess the type of 
fruits, vegetables, and legumes eaten in the past 
month and their frequency. 
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• Exercise & Physical Activity. Physical activity is 
assessed by the Exercise/Physical Activity module of 
the Behavioral Risk Factor Surveillance System [51]. 
Eight items assess the type, frequency, and duration 
of exercise/physical activity in the past month. 

• Health self-efficacy. The Patient Activation Measure 
(PAM) is a 13-item self-assessment of an individual's 
knowledge, skill, willingness, and confidence in 
managing their health and health care (activation) 
[52,53]. Scores stratify patients into four levels of 
health care activation. 

• Stress. Several aspects of stress are captured. The 
Perceived Stress Scale (PSS) [54] is a brief measure 
that assesses the degree to which situations in one's 
life are appraised as stressful in a global manner. 
Items tap how unpredictable, uncontrollable, and 
overloaded respondents find the stress in their lives 
to be. Two single items, found to strongly correlate 
with established multi-item measures, are used to 
tap the psychological response to stress and the 
amount of stress experienced in the past year [55]. 

• Sleep quality. Sleep quality is assessed by the 9-item 
Pittsburgh Sleep Quality Index (PSQI) [56]. The PSQI 
measures the quality and patterns of sleep over the 
past 1 month. It differentiates "poor" from "good" 
sleep by inquiring about seven areas: 1) subjective 
sleep quality, 2) sleep latency, 3) sleep duration, 
4) habitual sleep efficiency, 5) sleep disturbances, 
6) use of sleeping medication, and 7) daytime 
dysfunction. 

• Patient satisfaction. The 2-item measure of patient 
satisfaction is from the Consumer Assessment of 
Healthcare Providers and Systems (CAHPS) Survey 
2.0 Adult visit version [50,57] . These items are not 
included at baseline. 

• Pain (global). Global rating of pain is assessed by a 
single-item pain visual analogue scale ipVAS) [58]. It 
measures severity of pain over the past week. 

• Fatigue. Both a global rating and specific symptoms 
of fatigue are captured. Global rating of fatigue is 
assessed by a single-item fatigue visual analogue 
scale ijVAS). It measures the severity of fatigue over 
the past week. The 9-item Fatigue Severity Scale 
(FSS) [59] assesses how fatigue affects motivation, 
exercise, physical functioning, carrying out duties, 
and interfering with work, family, or social life in the 
past 1 week. 

• Depression. The 2-item ultra-brief Patient Health 
Questionnaire (PHQ-2) [60] is used to assess core 
symptoms of major depression. The items ask how 
often over the last 2 weeks participants were bothered 
by 1) having little interest or pleasure in doing things 
(anhedonia), and 2) feeling down, depressed, or 
hopeless (depressed mood). Scores of > 3 have a 



sensitivity of 83 % and a specificity of 92 % for major 
depression. Patients with scores of > 3 will also receive 
the full PHQ-9 (below). 

• Generalized Anxiety Disorder. The 2-item ultra-brief 
Generalized Anxiety Disorder questionnaire (GAD-2) 
[61] is used to assess core symptoms of anxiety 
disorder. The items ask patients how often over 
the last 2 weeks they were bothered by 1) feeling 
nervous, anxious, or on edge and 2) not being 
able to stop or control worrying. Scores of > 3 
have a sensitivity of 86 % and a specificity of 83 % 
for Generalized Anxiety Disorder. Patients with 
scores of > 3 will also receive the full GAD-7 
below. 

• Health-related Quality of Life. Health-related quality 
of life is assessed by the 12-Item Short Form Medical 
Outcomes Survey (SF-12) [62]. The SF-12 is a general 
measure of health status and well-being that 
assesses physical functioning, role limitations due to 
physical health, bodily pain, general health, energy/ 
fatigue, social functioning, and role limitations due 
to emotional problems and mental health in the 
past 4 weeks. 

• Psychological Well-being. Psychological well-being is 
assessed by the 5-item World Health Organization 
Well-Being Index (WHO-5) [63]. The WHO-5 
assesses subjective quality of life as a dimension 
separate from social disability and asks about 
dimensions such as positive mood, vitality, and 
interest in life in the past 2 weeks. 

• Health-related work productivity (absenteeism/ 
presenteeism). Health-related work productivity is 
assessed by the 6-item Work Productivity and 
Activity Impairment questionnaire (WPAI) [64]. The 
WPAI is a self-administered instrument to assess the 
impact of disease on absenteeism (work time missed), 
presenteeism (reduced on-the-job effectiveness), and 
impairment in daily activity. 

• Healthcare utilization. Four items based on Ritter 

et al. (2001) [65] ask about use of healthcare services 
in the prior period — from primary care providers 
outside of UAIHC, from specialists, from 
emergency rooms or urgent care centers, and 
overnight hospital stays. 

Self-report instruments - condition specific 

For participants diagnosed by UAIHC with any of the 
identified study clinical conditions (low back pain, 
fibromyalgia, depression and anxiety), one or more of 
the following questionnaires will be added: 

• Fibromyalgia Impact Questionnaire Revised - FIQR. 
The 21-item FIQR is a patient report of symptoms 
associated with fibromyalgia in three domains: 
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physical functioning (9 items), overall impact 
(2 items), and symptoms (10 items) [66]. 

• Roland-Morris Disability Questionnaire - RMDQ. 
The RMDQ is a patient report of physical disability 
due to low back pain (Roland & Morris, 1983). The 
RMDQ contains 24 yes/no items focused on 
limitations in walking, bending over, sitting, lying 
down, dressing, sleeping, self-care, and daily 
activities. 

• Patient Health Questionnaire 9 - PHQ-9. To assess 
clinical depression, the PHQ-9 [67,68] will be used 
with patients if the PHQ-2 screening score is > 3. 

• Generalized Anxiety Disorder 7 - GAD-7. When 
scores on the GAD-2 exceed the cut point of > 3 on 
the New Patient Intake Form, the 7-item GAD-7 
will be administered [69]. 

Data collected from medical records 

These data will be collected at 3, 6, and 12 months. A 
medical records abstraction form will be used to collect 
the following from the patient chart for the period since 
the last outcomes data collection point. The results of 
any of the following lab tests will be captured: fasting 
glucose, lipid panel, thyroid stimulating hormone (TSH), 
glycosylated hemoglobin (HbAlc), inflammatory markers 
(erythrocyte sedimentation rate and high-sensitivity 
C-reactive protein), vitamin D status, and measures of 
anemia (hemoglobin, hematocrit, and mean corpuscular 
volume). Also captured will be each patient's diagnosis 
(es), systolic and diastolic blood pressure, heart and re- 
spiratory rates, weight, waist circumference, body fat 
composition, smoking status, and whether the patient is 
on hypoglycemic, dyslipidemia, or hypertension medica- 
tion. Composite scores (10-year cardiovascular disease 
(CVD) event risk [70], 10-year CVD mortality risk [71], 
and metabolic syndrome diagnosis [72]) will be captured 
from the chart when recorded, or will be calculated. Forced 
vital capacity and forced expiratory volume will be captured 
for participants with pulmonary disease. 

Health Insurance Claims Data 

Annually, the UA Study Coordinator will create a coded, 
numeric list of study participants, who are employees or 
adult dependents of participating employer groups, to 
send to their health plan. This list will enable the health 
plan to generate and return a file of otherwise de- 
identified health claims data. Study participants will be 
identifiable (a partial waiver of protected health informa- 
tion data was granted by the UA Institutional Review 
Board) so that healthcare utilization data can be matched 
to their other study data, and the remainder of the file will 
be used to generate the matched control groups. The ini- 
tial data request will cover one year prior and one year 



after clinic enrollment for study participants, and a com- 
parable 2-year period for non-participants. 

Fidelity monitoring measures 
Patient feedback questionnaire 

Patient experiences of the various dimensions of integra- 
tive medical care will be assessed for their same-day care 
by a 51 -item "How is UAIHC doing?" Patient Version 
questionnaire developed for the fidelity study. Dimensions 
of the questionnaire include access to care, whole person 
care, promotion of self-care and wellness, practitioner 
communication style, shared decision-making, trust in the 
practitioner, perceived practitioner empathy, perceived 
health partnership, and socio-demographic information. 
Items were derived from established measures of primary 
care quality: the Consumer Assessment of Healthcare 
Providers and Systems (CAHPS) Surveys 2.0 Adult 
visit and Patient-Centered Medical Home versions [50,57], 
the Ambulatory Care Experiences Survey (ACES) [73], and 
the Consultation and Relational Empathy measure (CARE) 
[74]. Several items were developed for the study to tap 
patient experiences of IM care not assessed in existing 
instruments. It is estimated that it will take 10-15 minutes 
for patients to complete this questionnaire. 

Patient chart extraction 

A number of items will be extracted from a random 
sample of 10 de-identified patient charts at the end of 
each fidelity measurement month for patients seen dur- 
ing that month. The clinic staff will generate a list of the 
medical record numbers of all enrolled patients seen 
during the last month and report the total number (N) 
to the research team. The research team will use a sim- 
ple computer-based randomization scheme to create an 
equal length list consisting of "select" and "don't select" 
statements. Clinic staff will then match these lists and 
pull and de-identify those charts whose medical record 
numbers match up with the "select" statements. The 
data to be extracted by the research team are intended 
to assess IM whole person care (e.g., a review of systems 
beyond those typically assessed in conventional care in- 
cluding domains such as spirituality, social support, fam- 
ily and work stressors, etc.), the patient-practitioner 
partnership agreement, the mix of conventional and CM 
therapies used, and lifestyle recommendations over the 
past month. Visit lengths are also recorded. 

Practitioner experiences survey 

Practitioner experiences working in the UAIHC will be 
assessed using the 27-item "How is UAIHC doing?" 
Practitioner Version questionnaire developed for the fi- 
delity study. Dimensions of the questionnaire assess the 
team care climate of the clinic, practitioners' stress and 
self-care practices, and practitioners' feedback on how 
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practicing in the UAIHC model may affect their clinical 
care. The Team Climate Inventory (TCI) [75], a single- 
item on practitioner burnout [76], as well as additional 
individual items to account for IM principles will be ad- 
ministered. It is estimated that it will take 10-15 minutes 
for staff to complete this questionnaire. 

Qualitative/case study data 

The patients who have been selected for and agreed to 
case study interviews will participate in a 20-30 minute 
semi-structured interview immediately after each of the 
outcomes data collection points. The patient's outcome 
results over time will be available to the interviewer, and 
the interview will focus on the patient's experience of 
care at UAIHC and what the patient believes has con- 
tributed to any health changes seen in the course of 
treatment. These interviews will be either done by phone 
or in person, and will be audiotaped and transcribed. 

Analysis 

Five sets of inter-related analyses are planned for these 
data. The first will be a descriptive study based on the 
demographics and baseline values of the first patients 
(e.g., first 200) who join the study. The purpose of this 
study and report is to document the characteristics, 
diagnoses, symptoms and expectations of those patients 
who choose to enroll in this integrative primary care 
clinic. Where possible, patient characteristics, diagnoses 
and symptoms will be compared to those seen nation- 
ally and statewide in primary care using data from the 
National Ambulatory Medical Care Survey (NAMCS) 
[77]. Basic descriptive statistics will be used. When 
comparisons are possible, will utilize t tests for inde- 
pendent samples for continuous data, and \ tests for 
frequencies. 

The second analysis will use the fidelity monitoring 
data to describe clinic operations and to describe how 
this IM primary care clinic developed over time. Since 
these are measures of the intervention itself (the clinic), 
by design the patient fidelity sample measurements over 
time will not be taken on the same individuals. There- 
fore, data will be analyzed using simple descriptive sta- 
tistics. The results will also be examined for trends using 
regression analysis with time (either continuous or as a 
set of dummy variables) as the main explanatory vari- 
able, and significant t tests for the time coefficient(s) as 
an indication of significant change over time in clinic 
operations. Components of the clinic model that change 
significantly over time and/or that contribute to the out- 
comes will be considered as covariates in the two out- 
comes studies described below [24]. 

The third analysis will involve data from the case study 
participants. The narrative data from the transcribed, 
audiotaped semi-structured interviews will be content 



analyzed using dedicated qualitative data analysis/text 
management software (Atlas-ti), and key concepts will 
be categorized into salient themes and patterns. Those 
results will be related to each participant's reported and 
recorded outcomes to look for insights into clinic im- 
pacts and operations. 

The fourth and fifth analyses will be the main outcome 
studies. The fourth study will be a prospective cohort 
study of the changes in outcomes from baseline to one 
year for all patients in the outcomes sample. The pri- 
mary outcome for this analysis will be health-related 
quality of life measured in three ways (SF-6D [78], and 
the physical and mental component summary scores, all 
derived from the SF-12), with the other outcomes as sec- 
ondary measures. The main comparison will be baseline 
to 12-month outcomes and the univariate analyses will 
use paired t tests for continuous data, and use the 
McNemar test for frequencies [79]. The data will also be 
examined for patterns of change over time in key out- 
comes, and the determinants of those patterns, using 
hierarchical linear modeling (HLM) techniques [80-82]. 
Fidelity measures with significant changes over time will 
be used as covariates [24] in these models, as well as 
other explanatory variables such as demographics, par- 
ticipants' membership level and baseline diagnoses. If 
participant numbers for each of the five targeted condi- 
tion allow, condition-specific outcomes will be examined 
for each. These same analyses will be applied to other 
secondary outcomes such as CVD event [70] and mor- 
tality [71] risk. 

The fifth analysis will focus on the subset of patients 
for which health insurance claims data are available and 
will utilize a matched comparison design. Prior to ana- 
lysis all claims data for each individual (treatment and 
comparison groups) will be aggregated (by summation) 
into per health plan member per month (PMPM) totals 
of paid amounts (the amount paid by the health plan to 
the practitioner) for each of the following cost categor- 
ies: total, inpatient, outpatient, emergency room, phys- 
ician, and pharmaceutical claims. Similarly, the numbers 
of outpatient visits, hospital stays and emergency depart- 
ment visits will also be aggregated for each individual. 
The primary outcome will be total cost. Two comparison 
groups from de-identified claims data will be created — 
one using simple matching (e.g., by age and gender) and 
one using propensity scores [49]. After the comparison 
groups have been selected, simple descriptive analyses will 
show average PMPM costs and healthcare utilization (in- 
cluding standard deviations and other measures of disper- 
sion) pre- and post- enrollment (or an equivalent date for 
the comparison groups) by gender, age, baseline diagnosis 
and by treatment and comparison group. Because cost 
data tends to be highly skewed, average PMPM costs and 
healthcare utilization pre/post enrollment, and the change 
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in costs and utilization pre/post between the treatment 
and each of the comparison groups will be reported as 
mean differences with confidence intervals calculated 
using bootstrap methods [83,84]. Because of the number 
of data points for each patient (e.g., up to 12 PMPM 
values pre-enrollment and 12 post-enrollment), HLM 
[80-82] will be used to estimate the pattern of change in 
healthcare utilization and costs over time, and to explore 
the determinants of those patterns. Fidelity measures with 
significant changes over time [24], demographics, par- 
ticipants' membership level and baseline diagnoses will 
be included as covariates in these models. Finally, changes 
in costs between groups will be compared to changes in 
patients' CVD risk. 

Results for the last two analyses will be reported fol- 
lowing the Strengthening the Reporting of Observational 
Studies in Epidemiology (STROBE) guidelines [85]. For 
all outcomes, the estimated treatment effects will be re- 
ported using their original outcome scale and including 
confidence intervals. 

Discussion 

There has been much written about the need for new 
models of medical care — ones that are more patient- 
centered, that can empower patients to make positive 
lifestyle changes, that can lower cost of healthcare, and 
that offer a sustainable financial model. This study has 
been designed to test whether an IM primary care prac- 
tice model (UAIHC) can meet these challenges. 

This study will face a number of limitations. For ex- 
ample, some of the self-report measures of fidelity have 
not been fully tested and validated. The data collection 
burden on patients might limit enrollment or increase 
dropout. Also, because this is a new clinic, member enroll- 
ment and study adoption rates are unknown and could fall 
short of predictions. This is especially true for the five tar- 
geted conditions; low prevalence will prevent analyses by 
condition. Obtaining a useable set of claims data is always 
challenging. Better outcomes (larger effect sizes) may be 
achieved were factors in the clinic subject to more control 
(e.g., patient homogeneity, or treatment consistency rather 
than patient-centered care), or if the clinic had already 
had time to mature into its final form. Also, at such an 
early phase of the clinic's development and management, 
the methodological rigor required to study this multi- 
faceted system of care may not be able to yield all of the 
desired information. Instead, this will be an evaluation of 
a newly designed and formed IM primary care clinic 
model in its natural state as it evolves. Some of the data 
collected during fidelity monitoring will also be useful in 
improving and optimizing clinic performance. The addition 
of more qualitative data collection is desirable to more 
closely examine patient experiences in the clinic and may 
be possible as funds and time become available. 



This protocol is designed to meet the methodological 
challenges involved in evaluating this complex interven- 
tion (IM model of adult primary care) with as much rigor 
as possible. It is anticipated that its results and the results 
of anticipated continuous evaluation of the UAIHC will 
contribute greatly to health services research in integrative 
medicine. 
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